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(54) Information processing apparatus and method 



(57) A moving image processing apparatus can per- 
form a zoom process for each frame image of moving 
image data in an digital zoom section (17). An image 
quality determining section (21a) determines an image 
quality of a frame image obtained through the zoom 
process in reproduction, when this zoom process is per- 
formed, and passes quality information representing its 
determined result to an MPEG CODEC (26). The MPEG 



CODEC (26) encodes the moving image data proc- 
essed in the digital zoom section (17) and a noise re- 
duction section (18), and stores the image data, and the 
quality information associated with each frame in a stor- 
age medium (28). In this manner, when the reproduction 
object data is processed to have some effect on the 
quality in reproduction, the data can be provided with 
the information regarding the quality in reproduction. 
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Description 

FIELD OF THE INVENTION 

[0001] The present invention relates to a technique for 
recording additive Information together with information 
to be reproduced and utilizing the additive information. 

BACKGROUND OF THE INVENTION 

[0002] Conventionally, there was offered means for 
recording information sensed by a sensor in an image 
device and the setting information set by a user along 
with a moving image, and enabling the moving image to 
be retrieved, reproduced or printed based on the infor- 
mation, as disclosed in Japanese Patent Laid-Open No. 
7-99605, Japanese Patent Laid-Open No. 7-200632, 
Japanese Patent Laid-Open No. 11-298849, Japanese 
Patent Laid-Open No. 2000-350234, and Japanese Pat- 
ent Laid-Open No. 2000-358206. For example, ON or 
OFF state and the magnifying power of the digital zoom 
described as the digital zoom information are utilized for 
retrieving a frame which has been magnified at a pre- 
determined digital zoom power, as disclosed in Japa- 
nese Patent Laid-Open No. 7-99605. 
[0003] However, even if the image device is operated 
in the same manner, the image quality obtained as a 
result of the process may be varied due to a difference 
in the processing method. For example, in the digital 
zoom, it is common practice that a part of an image cap- 
tured by the image pickup element is cut out and en- 
larged. With this method, in enlargement, insufficient 
pixels are interpolated from neighboring pixels, resulting 
in degraded image quality. However, a method of pre- 
venting the degradation in the image quality that is 
caused by an interpolation process of pixels was pro- 
posed (Japanese Patent Laid-Open No. 7-67032). Also, 
there is an image device in which the digital zoom sys- 
tem is changed depending on the magnifying power. In 
this way, with the different processing method, there oc- 
curs a difference in the image quality after the digital 
zoom process. And if the information regarding the im- 
age quality caused by a difference in the processing 
method is utilized in printing the frame image, for exam- 
ple, it is possible to avoid printing the frames of bad im- 
age quality. With the prior techniques, however, only the 
information sensed by the sensor and the setting infor- 
mation set by the user could be described as the additive 
information, and therefore, the utilization of image qual- 
ity as discussed above (or determination of the image 
quality) was impossible. 

SUMMARY OF THE INVENTION 

[0004] The present invention has been made in the 
light of the above-mentioned problems. An embodiment 
of the invention provides the data to be reproduced with 
information regarding the image quality in reproduction, 



when the data is processed to have some effect on the 
image quality in reproduction. 

[0005] An embodiment of the invention enhances the 
operability, employing the information regarding the im- 
5 age quality in reproduction. 

[0006] An information processing apparatus accord- 
ing to this invention includes: processing means for 
processing data intended for reproduction; determina- 
tion means for determining a quality of the data in repro- 
10 duction after processing the data by the processing 
means to generate quality information representing its 
quality, and storage means for storing the data proc- 
essed by the processing means and the quality informa- 
tion obtained by the determination means in association 
15 with each other. 

[0007] Also, another information processing appara- 
tus according to this invention includes: acquisition 
means for acquiring reproduction object data and the 
quality information of each portion of the reproduction 
20 object data from the data composed of the reproduction 
object data and the quality information representing the 
quality of the reproduction object data in reproduction, 
the quality information being associated with the repro- 
duction object data; and display means for displaying 
?5 the reproduction object data or the information indicat- 
ing the reproduction object data based on the quality in- 
formation acquired by the acquisition means. 
. [0008] An information processing method according 
to this invention includes: a processing step of process- 
30 ing data intended for reproduction; a determination step 
of determining a quality of the data in reproduction after 
processing the data at the processing step to generate 
the quality information representing its quality; and a 
storing step of storing the data processed at the 
w processing step and the quality information obtained at 
the determination step in association with each other. 
[0009] Also, another information processing method 
according to this invention includes: an acquisition step 
of acquiring reproduction object data and associated 
l o quality information from the data including the reproduc- 
tion object data and the quality information representing 
the quality of the reproduction object data in reproduc- 
tion, the quality information being associated with the 
reproduction object data; and a display step of display- 
5 ing the reproduction object data or the information indi- 
cating the reproduction object data based on the quality 
information acquired at the acquisition step. 
[0010] Features and advantages of the embodiments 
will be apparent from the following description taken in 
o conjunction with the accompanying drawings, in which 
like reference characters designate the same or similar 
parts throughout the figures thereof. 

BRIEF DESCRIPTION OF THE DRAWINGS 

5 

[0011] The accompanying drawings, which are incor- 
porated in and constitute a part of the specification, il- 
lustrate embodiments of the invention. 
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[0012] FIG. 1 is a block diagram showing a configu- 
ration of a moving image processing apparatus to which 
the present invention is applicable. 
[0013] FIG. 2 is a block diagram showing a configu- 
ration of an image device according to an embodiment 5 
of the invention. 

[0014] FIG. 3 is a block diagram showing a configu- 
ration of a system control section 21 . 
[001 5] FIG. 4A is a flowchart showing an digital zoom 
and a noise reduction control processing procedure. 
[001 6] FIG. 4B is a flowchart showing the digital zoom 
and the noise reduction control processing procedure. 
[0017] FIGS. 5A and 5B are explanatory views for ex- 
plaining a zooming process with an digital zoom section 
17. 

[001 8] FIG. 6 is a diagram showing a format example 
of digital zoom information in this embodiment. 
[0019] FIG. 7 is a block diagram showing a control 
configuration of a printer according to this embodiment 
of the invention. 

[0020] FIG. 8 is a block diagram showing a functional 
configuration of the printer. 

[0021] FIG. 9 A and 9B are views illustrating an exam- 
ple of a user interface with a display on a printer side. 
[0022] FIG. 1 0 is a flowchart showing an operation ex- 
ample of the printer. 

[0023] FIG. 11 is a diagram for explaining a way of 
recording the digital zoom information in each interval. 
[0024] FIGS. 12A to 12C are views showing another 
example of the user interface with the display on the 
printer side. 

[0025] FIGS. 1 3 A to 1 3C are views showing a display 
example of the user interface with a moving image ed- 
iting device. 

DETAILED DESCRIPTION OF THE PREFERRED 
EMBODIMENTS 

[0026] Preferred embodiments of the present inven- 
tion will now be described in detail in accordance with 
the accompanying drawings. 

[0027] FIG. 1 is a block diagram showing a configu- 
ration of a moving image processing apparatus to which 
the present invention is applicable. In FIG. 1 , reference 
numeral 1 01 0 denotes an image device for picking up 
a moving image, 1020 denotes a storage medium for 
storing the moving image, and 1030 denotes a printer 
for printing a specified frame image extracted from the 
moving image contained in the storage medium 1020. 
[0028] Next, the configuration and its control method 
of the image device 1 010 according to the invention, and 
the storage medium 1020 will be described below. FIG. 
2 is a block diagram showing a configuration of the im- 
age device according to the invention. In FIG. 2, refer- 
ence numeral 11 denotes a zoom lens for forming an 
image of the subject, 12 denotes a diaphragm for ad- 
justing a quantity of light, 13 denotes an image pickup 
element composed of a CCD for converting input light 
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into an electric signal, and 14 . denotes a sample hold 
AGC circuit section for performing the sample hold and 
gain adjustment. 

[0029] Reference numeral 15 denotes an A/D con- 
verter for performing an analog/digital conversion (A/D 
conversion), and 16 denotes a video signal processor 
for performing the signal processing to produce a video 
signal. Reference numeral 17 denotes an digital zoom 
section having a field memory for performing the digital 
zoom by cutting out a video and executing the enlarge- 
ment and interpolation process on the video. Reference 
numeral 1 8 denotes a noise reduction (NR) section hav- 
ing afield memory for removing the noise from the video 
signal. The noise reduction (NR) section 18 is com- 
posed of a cyclic noise reduction circuit. Reference nu- 
meral 1 9 denotes a lens position detecting section for 
detecting the position of a lens, and 20 denotes a lens 
driver section for driving the lens. 
[0030] Reference numeral 21 denotes a system con- 
troller for controlling an entire image device. FIG. 3 is a 
block diagram showing a configuration of the system 
controller 21. The system controller 21 has a well known 
CPU 31 , a ROM 32, a RAM 33, an I/O interface 34, and 
a bus 35. The ROM 32 stores a program to be executed 
by the CPU 31 and table values. The system controller 
21 has an image quality determining section 21 a for de- 
termining the image quality in reproduction based on the 
processed contents by the digital zooming section 17, 
as will be described later in connection with the flow- 
charts of FIGS. 4A and 4B. 

[0031 ] Reference numeral 22 is a zoom operation key 
section having the operation keys for the zoom opera- 
tion. Reference numeral 23 denotes a wide-angle key 
for operating the zoom in a wide-angle direction (wide- 
angle = image reduction). Reference numeral 24 de- 
notes a telephoto key for moving the lens in a telephoto 
direction (telephoto = image enlargement). The wide- 
angle key 23 and the telephoto key 24 are interlock keys 
of the seesaw type, in which an output signal indicating 
which key is pressed with what extent of pressure is out- 
put to the system controller 21 . 

[0032] Reference numeral 26 denotes an MPEG CO- 
DEC for encoding the video data into the MPEG format. 
Reference numeral 27 denotes a recording section 
composed of a recorder for recording the information by 
driving the recording medium. Reference numeral 28 
denotes the recording medium, such as an optical disk, 
a magnetic disk, an optical magnetic disk, a magnetic 
tape, a hard disk, and a non-volatile semiconductor 
memory (flash memory). 

[0033] The image pickup operation of the image de- 
vice according to the embodiment with the above con- 
stitution will be briefly described below. A light from the 
subject that has received the light from the zoom lens 
11 has its light quantity adjusted by the diaphragm 12, 
and forms an image on the plane of the image pickup 
element 13. And after the image is converted into an 
electric signal by the image pickup element 1 3, the elec- 
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trie signal is sampled and gain-adjusted by the sample 
hold AGC section 14. The signal from the sample hold 
AGC section 14 is A/D converted by the A/D converter 
15, and input into the video signal processor 1 6. 
[0034] In the video signal processor 1 6, processings 
such as aperture correction, gamma correction and 
white balance correction are performed against an input 
signal every brightness and color component to gener- 
ate the video signal which is output to the digital zoom 
section 1 7. The digital zoom section 1 7 cuts out an im- 
age, and performs the enlargement and interpolation 
process using the field memory, so that the ratio of out- 
put video to input video may be the magnifying power 
designated by a control signal from the system controller 
21. An output video from the digital zoom section 17 is 
input into the noise reduction section 18. 
[0035] The noise reduction section 18 removes the 
noise from the video signal by being controlled based 
on a control signal from the system controller 21. The 
video signal whose noise has been removed is output 
to the MPEG CODEC 26. The MPEG CODEC 26 en- 
codes the input video signal. At this time, the MPEG CO- 
DEC 26 stores the control information (hereinafter re- 
ferred to as digital zoom information) input from the sys- 
tem controller 21 , together with the encoded video sig- 
nal, and outputs it to the recording apparatus connected 
to the MPEG CODEC 26. A method of storing the digital 
zoom information will be described later. 
[0036] The system controller 21 controls each section 
of the image device, and controls the lens driving section 
20 or the digital zoom section 1 7 to move the zoom lens 
11 in the wide-angle or telephoto direction, if the wide- 
angle key 23 or the telephoto key 24 of the zooming op- 
eration key section 22 is pressed. Also, the system con- 
trolled determines where the zoom lens 11 is located, 
the telephoto end (telephoto endmost point), the wide- 
angle end (wide-angle endmost point), or some place 
between the telephoto end and the wide-angle end 
(zoom magnification), based upon a lens position detec- 
tion signal from the lens position detecting section 19. 
Here, the function of enlarging an image picked up by 
the lens optical system is called an optical zoom, and 
the function of enlarging the image in vertical or horizon- 
tal direction through the electronic signal processing 
without relying on the lens optical system is called an 
digital zoom. 

[0037] The system controller 21 usually performs the 
optical zoom by driving the zoom lens 11 in a range of 
magnifying power that is settable in the optical zoom, 
and electronically enlarges the image by the digital 
zoom under the control of the digital zoom section 1 7, 
after the zoom lens 11 reaches the telephoto end. In this 
embodiment, the optical zoom has a maximum magni- 
fying power of twelve, and the digital zoom has a max- 
imum magnifying power of four, whereby the zoom op- 
eration having a high magnifying power of forty eight can 
be performed. 

[0038] Also, the system controller 21 switches the 
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control value of the noise reduction process according 
to whether the optical zoom or the digital zoom is in op- 
eration, and outputs a control signal to the noise reduc- 
tion section 1 8. Thereby, the effective amount of noise 
5 reduction, namely, the removal amount of noise is con- 
trolled. The control value to be switched is a cyclic factor 
K. 

[0039] The switching of the control value for the noise 
reduction section 1 8 may be also made based on the 
10 brightness of the subject. For example, with a lower 
brightness of the subject, the S/N ratio of the video sig- 
nal is worse, whereby the control value is switched to 
increase the effective amount of noise reduction to 
make up for it. 

15 [0040] Further, the system controller 21 controls the 
diaphragm 12, the sample hold AGC section 14, and an 
electronic shutter (not shown in FIG. 2) to maintain the 
generated video signal at a predetermined level value 
for the amount of light from the subject, whereby the 
20 brightness of the subject is judged comprehensively, 
employing the diaphragm value, the AGC gain and the 
electronic shutter speed for use in the exposure control. 
[0041] Next, the control for the optical zoom, the dig- 
ital zoom and the noise reduction by the system control- 
's Ier21 will be described below. FIGS. 4A and 4B are flow- 
charts showing an digital zoom and noise reduction con- 
trol processing procedure. This processing program is 
stored in the ROM 32 within the system controller 21 , 
and executed by the CPU 31 . 
so [0042] First of all, the system controller21 determines 
whether or not the telephoto key 23 is turned on (step 
S401). If the telephoto key 24 is turned on, it is deter- 
mined whether or not the position of the zoom lens 11 
is at the telephoto end (optical zoom magnifying power 
35 is twelve) (step S402). If the position of the zoom lens 
11 is at the telephoto end, the zoom lens 11 is stopped 
at the telephoto end (step S403). And it is determined 
whether or not the digital zoom magnifying power by the 
digital zoom section 17 is equal to a maximum of four 
40 (step S404). If not, the digital zoom section 17 is con- 
trolled to increase the magnifying power at a predeter- 
mined variable power ratio (from step S405 to step 
S409). On one hand, if the digital zoom ratio is equal to 
the maximum of four, the digital zoom magnifying power 
45 is kept four (steps S410, S411). In this embodiment, the 
optical zoom magnifying power is the maximum of 
twelve, and the digital zoom magnifying power is the 
maximum of four, but other magnifying powers may be 
used. 

so [0043] The processing procedure of the digital zoom 
section 1 7 from step S405 to step S409 will be described 
below with reference to FIG. 5 that is a concept view for 
enlargement by the digital zoom. To enhance the image 
quality in the digital zoom, the image device of this em- 

55 bodiment involves different digital zoom methods, de- 
pending on whether the magnifying power is in a range 
from one to two or from two to four. Thus, it is firstly de- 
termined whether or not the digital zoom magnifying 
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power is two or greater (step S405). 
[0044] When the magnifying power is less than two (i. 
e., from one time to twice), the procedure goes to step 
S406, where the reduction ratio of CCD image is de- 
creased in accordance with an digital zoom magnifying 
power. Referring to FIG. 5 A, this process will be de- 
scribed below. 

[0045] On the image pickup element 1 3, a range twice 
as large as a utilization area 502 that is equal to one 
screen to be actually recorded (frame or field, but the 
frame is used here for the simplification of explanation) 
is employed (501). When the digital zoom is not used (i. 
e., the digital zoom is at one magnifying power), an im- 
age pickup element image 501 is reduced half (506) into 
a size of 502. For example, if 502 is 320x120 pixels, 
501 is 453x 1 70 pixels of double size. Though the image 
quality is slightly degraded by reduction, the number of 
pixels for the image pickup element is employed to be 
larger than that of the utilization area to offset the deg- 
radation. 

[0046] And, in a range of magnifying power from one 
to two, the reduction ratio is changed in accordance with 
the magnifying power. For example, when the digital 
zoom has a magnifying power of one and a half, the im- 
age is reduced into a size as large as 392x147 pixels 
that is (1 .5/2) size by the thinning process of the pixels 
(507). That is, when the digital zoom has a magnifying 
power of N, the image as large as 501 is reduced .into 
N/2. And an area (502) of 320x120 pixels around the 
center of image is cut out and passed to the noise re- 
duction section 18. In this manner, in the range of mag- 
nifying power from one to two, the digital zoom is imple- 
mented by changing the reduction ratio of CCD image 
in accordance with the magnifying power. 
[0047] The reduction ratio is decreased as the mag- 
nifying power of digital zoom is increased. As a result, 
the number of pixels of the image pickup element used 
for digital zoom processing (the number of pixels used 
to form an image in the area 502) is decreased. How- 
ever, since the degradation due to the reduction process 
is also decreased, the image quality can be kept almost 
equal. That is, in the digital zoom processing with the 
magnifying powerfrom one to two, the image quality can 
be kept almost equivalent to that when the digital zoom 
is not in use. Thus, the image quality determination sec- 
tion 21 a of the system controller 21 determines that the 
reproduced image quality is excellent, and set Q to 
HIGH. This information, together with the digital zoom 
magnifying power, is passed to the MPEG CODEC 26, 
and stored in a format as will be described later (step 
S407). 

[0048] On the other hand, at step S405, if the digital 
zoom has a magnifying power of two or more (i.e., the 
digital zoom with the magnifying power of two to four), 
the procedure goes to step S408. At step S408, the en- 
largement ratio of image in the memory is increased in 
accordance with the digital zoom magnifying power. Re- 
ferring to FIG. 5B, this process will be described below. 



[0049] In the case where the digital zoom has a mag- 
nifying power from two to four, an image of the image 
pickup element is taken into the memory without reduc- 
tion (508), a utilization area as large as 320x120 pixels 
5 (hereinafter referred to as memory image) is cut out from 
around the center of image in the memory, and enlarged 
in accordance with the digital zoom magnifying power 
(509). The image (510) corresponding to the utilization 
area as large as 320x120 pixels is cut out again from 
the image (511) after enlargement and passed to the 
noise reduction section 18. In enlargement, insufficient 
pixels are required to be interpolated from the neighbor- 
ing pixels, resulting in degraded image quality. Thus, the 
image quality determination section 21a of the system 
controller 21 determines that the reproduced image has 
a lower image quality and sets Q to LOW. This informa- 
tion, together with the digital zoom magnifying power, is 
passed to the MPEG CODEC 26, and stored in a format 
as will be described later (step S409). 
[0050] At step S404, if the digital zoom has a magni- 
fying power of four time, the enlargement ratio of mem- 
ory image is fixed at a maximum of twice, because the 
digital zoom magnifying power gets to the maximum 
(step S410). At this time, since the image quality is de- 
graded, the image quality determination section 21a of 
the system controller 21 set Q to LOW. This information, 
together with the digital zoom magnifying power, is 
passed to the MPEG CODEC 26, and stored in a format 
as will be described later (step S411). . . . 
[0051] On the other hand, if the position of the zoom 
lens 11 is not at the telephoto end at step S402, the lens 
driving section 20 (in telephoto direction) is controlled 
so that the magnifying power of the zoom lens 11 may 
be increased at a predetermined variable power ratio 
(step S412). And the reduction ratio of image on the im- 
age pickup element is fixed at one-half (step S413). 
[0052] On one hand, if the telephoto key 24 is not 
turned on at step S401 , it is determined whether or not 
the wide-angle key 23 is turned on (step S414). If the 
wide-angle key 23 is turned on, it is determined whether 
or not the digital zoom magnifying power by the digital 
zoom section 17 is a minimum value (one time) (step 
S415). 

[0053] If the digital zoom magnifying power is the min- 
imum value (one time), the digital zoom section 17 is 
controlled so that the magnifying power of digital zoom 
be kept one time, whereby the digital zoom section fixes 
the reduction ratio of image on the image pickup ele- 
ment at one-half (step S416). Then, it is determined 
whether or not the position of the zoom lens 11 is at the 
wide-angle end (optical zoom magnifying power of one) 
(step S417). If so, the zoom lens 11 is stopped at the 
wide-angle end (step S418). On the other hand, if the 
position of the zoom lens 11 is not at the wide-angle end, 
the lens driving section 20 is controlled so that the mag- 
nifying power of the zoom lens be reduced at a prede- 
termined variable power ratio (wide-angle direction driv- 
ing) (step S419). 
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[0054] On one hand, if the magnifying power of the 
digital zoom is not one at step S415, the digital zoom 
section 1 7 is controlled so that the magnifying power of 
the zoom lens be reduced at a predetermined variable 
power ratio (from step S420 to step S425). As described 
above, since the method of the digital zoom is different 
depending on whether or not the magnifying power is 
greater than two, it is firstly determined whether or not 
the digital zoom has a magnifying power of two or more 
(step S420). At the magnifying power of two or more, 
the enlargement ratio of memory image is decreased in 
accordance with the magnifying power of digital zoom 
(step S421). For the above reason, the image quality of 
reproduced image is degraded in this state, accordingly, 
the image quality determination section 21a of the sys- 
tem controller 21 sets Q to LOW. This information, to- 
gether with the digital zoom magnifying power, is passed 
to the MPEG CODEC 26, and stored in a format as will 
be described later (step S422). On the other hand, if it 
is determined that the digital zoom magnifying power is 
less than two, the reduction ratio of image on the image 
pickup element is increased in accordance with the 
magnifying power of digital zoom (step S423). And the 
image quality of reproduced image can be kept high in 
this state, whereby the image quality determination sec- 
tion 21a of the system controller21 sets Q to HIGH. Un- 
der the control of the system controller 21 , this informa- 
tion, together with the digital zoom magnifying power, is 
passed to the M PEG CODEC 26, and stored in a format 
as will be described later (step S424). And the zoom lens 
11 is stopped at the telephoto end (step S425). 
[0055] On one hand, at step S414, if the wide-angle 
key 23 is not turned on, the digital zoom section 17 is 
controlled so that the magnifying power of digital zoom 
is not variable but fixed (from step S426 to step S428). 
The digital zoom section 1 7 determines whether or not 
the digital zoom has a magnifying power of two or more 
(step S426). if the magnifying power is less than two, 
the reduction ratio of image on the image pickup ele- 
ment is fixed (S427). On the contrary, if it is two or more, 
the enlargement ratio of memory image is fixed (S428). 
And the lens driving section 20 is controlled so that the 
magnifying power of the zoom lens 11 is not variable but 
fixed, whereby the zoom lens 1 1 is stopped (step S429). 
[0056] And in a state where the digital zoom section 
17 and the zoom lens 11 are stopped, the noise reduc- 
tion section 1 8 is controlled to place the control mode of 
noise reduction in a zoom stop mode (step S430). On 
one hand, in a state where either one of the digital zoom 
section 1 7 and the zoom lens 1 1 is operated, the noise 
reduction section 1 8 is controlled to place the control 
mode of noise reduction in a zoom operation mode (step 
S431). Then, once the system controller 21 ends this 
process, it controls each section of the image device to 
repeat the above process from step S401 to step S431 
again. 

[0057] Next, a method of recording the digital zoom 
information together with the moving image will be de- 



scribed below. The digital zoom information of this em- 
bodiment contains the image quality information Q that 
is set up at steps S407, S409, S411 , S422, and S424. 
[0058] In recording the digital zoom information, a 

5 method of utilizing an unused area of the management 
information for each frame of the image data may be 
employed. More specifically, for an MPEG-2 stream, the 
digital zoom information is stored in the user data area 
of a picture header orthe user data area of a GOP head- 

10 er. Or the image quality information may be stored in 
correspondence to an identifier (frame ID) for designat- 
ing each frame. 

[0059] Next, the format for recording the digital zoom 
information will be described below. FIG. 6 is a diagram 
15 showing a format example of digital zoom information 
in this embodiment. In Digital Zooming Quality, the qual- 
ity (the content of image quality information Q) corre- 
sponding to the digital zoom method is described. If an 
image of high quality is obtained, it is described as High, 

20 or if the image quality is degraded, it is described as 
Low. For example, when the digital zoom is used at a 
magnifying power of one to two, High is written, orwhen 
it is used at a magnifying power of two to four, Low is 
written in this embodiment. 

25 [0060] If the image device is equipped with a plurality 
of digital zoom methods, and the image without degra- 
dation in the image quality is obtained by any of the dig- 
ital zoom methods, High may be always written in Digital 
Zooming Quality. Employing the image device having a 

30 single digital zoom method, High or Low may be de- 
scribed in Digital Zooming Quality, depending on the de- 
gree of degradation in the image quality. 
[0061] In FIG. 6, Digital Zoom Mode is a mode of an 
digital zoom, in which Disable or Enable of Digital Zoom 

35 Mode is written. Magnification is the digital zoom mag- 
nification described in a form of factors which satisfy an 
expression of FIG. 6. For the frames in which the digital 
zoom is not used, the information is not described. 
[0062] The configuration and operation of a printer 

40 1 030 according to this embodiment will be described be- 
low. 

[0063] FIG. 7 is a block diagram showing the control 
configuration of the printer according to this embodi- 
ment, in FIG. 7, reference numeral 701 denotes a CPU 

45 that makes the control of the moving image processing 
apparatus of this embodiment. Reference numeral 702 
denotes a ROM for storing a boot program or various 
sorts of data to be employed at the time of starting up 
this processing apparatus. Reference numeral 703 de- 

50 notes a RAM for storing a control program to be execut- 
ed by the CPU 701 , and providing a working area for the 
CPU 701 to perform various sorts of control. Reference 
numeral 704 denotes a keyboard, and 705 denotes a 
mouse for providing various input operation environ- 

55 ments for the user. 

[0064] Reference numeral 706 denotes an external 
storage unit, which is composed of a hard disk, a floppy 
disk, an optical disk, a magnetic disk, an optical mag- 
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netic disk, or a magnetic tape. Reference numeral 707 
denotes a display unit composed of a display to provide 
the results to the user. Reference numeral 708 denotes 
a network interface for enabling the communication with 
various devices on the network. Reference numeral 709 
denotes a printing apparatus composed of a printer. 
[0065] FIG. 8 is a block diagram showing the function- 
al configuration of the printing apparatus. Reference nu- 
meral 801 denotes a recording medium that corre- 
sponds to the recording medium 28 in FIG. 2. A reading 
section 802 reads a moving image and the digital zoom 
information on the recording medium 801. Reference 
numeral 803 denotes a frame selecting section for de- 
ciding a frame in the moving image to be printed. Ref- 
erence numeral 804 denotes a user operation section 
composed of the keyboard 704 and the mouse 705 to 
instruct the selection of frame. Reference numeral 805 
denotes a display section for displaying the frame as a 
print candidate on the display unit 707. Reference nu- 
meral 806 denotes a printing section composed of the 
printing apparatus 709 to print the frame image selected 
by the frame selecting section 803. 
[0066] FIGS. 9 A and 9B illustrate an example of a us- 
er interface with the display unit on the printing appara- 
tus side. Reference numeral 901 denotes a display ar- 
ea. Reference numeral 902 denotes a thumb-nail dis- 
play area in which thumb-nail images consisting of re- 
duced frame images are arranged in the time series or- 
der, starting from the upper left side, from left to right, 
and from upper to lower side. Reference numeral 903 
denotes a scroll bar for scrolling the thumb-nail images 
for access, when all thumb-nail images can not be dis- 
played in the thumb-nail area 902 at a time. Reference 
numeral 904 denotes a pointer that the user can manip- 
ulate, employing the mouse 705, to issue various kinds 
of instruction. 

[0067] FIG. 10 is a flowchart showing an operation ex- 
ample of the printing apparatus. This process is per- 
formed by the CPU 701 that executes the control pro- 
gram stored in the ROM 702. 

[0068] First of all, the reading section 802 reads a 
moving image that the user has designated via the user 
operation section 804 from the storage medium 801 
(step S1001). Then, the frame selecting section 803 
reads the frames in the moving image successively and 
displays them in thumb-nail (from step S1002 to step 
S 1 006). Though in this embodiment, each of the frames 
is made thumb-nail, the frames may be made thumb- 
nail for every predetermined number of frames. 
[0069] Step S1 002 is an end determination, whereby 
the process from step S1 003 to step S1 006 is repeated 
till ail the frames in the moving image are processed. At 
step S1003, the frame image is reduced into an appro- 
priate size of thumb-nail image to display an arrange- 
ment of frame images on the screen, as shown in FIG. 
9A. Then, Digital Zooming Quality of the digital zoom 
information corresponding to the frame image is 
checked. It is determined whether or not Digital Zooming 



Quality is Low (step S1 004). If it is Low, it is determined 
that the image quality is degraded. Thus, to invite the 
user's attention, the thumb-nail of the frame image 
whose quality is degraded is displayed in a thumb-nail 

5 display area 902, along with the x sign as shown in FIG . 
9A (step S1005). On the other hand, if it is determined 
at step S1004 that Digital Zooming Quality is not Low 
(including the cases where no zoom information exists), 
the thumb-nail is directly displayed (step S1006). 

10 [0070] By repeating the above process, the thumb- 
nails are displayed in the thumb-nail display area 902. 
Though not shown in the flowchart, if the maximum 
number of displaying the thumb-nails in the thumb-nail 
display area 902 is exceeded, the thumb-nails that can 

15 not be displayed are stored in the memory, and when 
the scroll bar 903 is operated, the corresponding thumb- 
nails are displayed in the thumb-nail display area 902. 
[0071] After the thumb-nails are displayed in the 
above manner, it is determined whether or not the user 

20 selects the thumb-nail to be printed using the pointer 
904 (step S1007). If selected, the frame image corre- 
sponding to the selected thumb-nail is printed in the 
printing section 806 at step S1 008. 
[0072] In the above manner, since the x sign is indi- 

25 cated with the thumb-nail image, it is clearly pointed out 
that the image quality of the frame image is degraded, 
whereby the user can utilize it as a yardstick in selecting 
, the frame image to be printed. Generally, the thumb-nail 
is the greatly reduced frame image, and it is difficult to 

30 determine whether or not the image quality is degraded 
from the thumb-nail image. According to this invention, 
the user can easily discriminate the image quality. 
[0073] While in the above embodiment the digital 
zoom information is added to each frame, it may be 

35 stored in a form of having association with each frame. 
For example, there is a method of storing it in the footer 
of amoving image file for each interval, as shown in FIG. 
11 . In FIG. 11 , the record is registered for each interval 
in which Digital Zooming Mode, Digital Zooming Quality, 

40 and Magnification are the same. Each interval is indicat- 
ed by a Start Point and an End Point. The Start Point 
and the End Point representing the interval can be cho- 
sen at will, such as the time information or the frame ID, 
so long as they are associated with the frame. 

45 [0074] While in this embodiment the digital zoom is 
operated after the zoom lens 11 reaches the telephoto 
end, of course, the digital zoom may be operated before 
reaching the telephoto end by providing an operation 
button for digital zoom. 

so [0075] While in this embodiment, the thumb-nail hav- 
ing the degraded image quality is displayed with the sign 
x , thereby inviting the user's attention, as shown in FIG. 
9A, it will be appreciated that only the thumb-nails with- 
out degradation in the image quality may be displayed, 

55 except for the thumb-nails of degraded image quality, 
as shown in FIG. 9B. 

[0076] In this embodiment, the image quality is High 
or Low, but if the image quality is subdivided into three 
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or more levels by the digital zoom processing method. 
For example, three levels of High, Middle and Low may 
be employed. Furthermore, the sign may be displayed 
in such a mannerthat image quality High is no sign, Mid- 
dle is A sign, and Low is x sign, as shown in FIG. 12A. 
Also, a display mode may be provided to enable various 
display of thumb-nails, such that all images are dis- 
played (FIG. 12A), the images of poor image quality are 
not displayed (FIG. 12B), or the images of excellent im- 
age quality are displayed (FIG. 12C). 
[0077] While in this embodiment the digital zoom in- 
formation is employed forthe printing apparatus, it may 
be utilized in various kinds of apparatuses, including a 
moving image editing device, a reproducing apparatus 
and a retrieval device. In any utilization form, it is pos- 
sible to realize at least the following three advantages: 
(1 ) inviting the user's attention to the frames of degraded 
image quality, (2) excluding the frames of degraded im- 
age quality from the use objects, and (3) displaying the 
frames in accordance with the degree of degradation in 
the image quality. 

[0078] As other apparatuses than the printing appa- 
ratus, the moving image editing device is exemplified. 
FIGS. 13A to 13C are views illustrating a display exam- 
ple of the moving image editing device. In FIG. 1 3A, ref- 
erence numeral 1301 denotes a bar representing an en- 
tire moving image file. Reference numeral 1304 denotes 
a position designation pointerthat is used in designating 
the period for editing. Reference numeral 1302 denotes 
a period where Digital Zooming Quality is Middle in the 
moving image 1301, and 1302 denotes a period where 
Digital Zooming Quality is Low. The user can avoid the 
period with degraded image quality in editing by refer- 
ring to the display of FIG. 1 3A. 

[0079] The periods (1 302 and 1 303) with degradation 
in the image quality may be excluded from the display, 
as shown in FIG. 1 3C, and the user may be inhibited to 
select those periods. In response to an instruction from 
the user, the images may be displayed at several levels, 
such that all images are displayed, as shown in FIG. 
13A, the images except forthe period with severe deg- 
radation in the image quality are displayed, as shown in 
FIG. 13B, or the images except for all the periods with 
degradation in the image quality are displayed, as 
shown in FIG. 13C. It is apparent that the above display 
is enabled by referring to Digital Zooming Quality corre- 
sponding to the frame. 

[0080] While in this embodiment, two apparatuses are 
provided, including the image device 1010 for adding 
the digital zoom information and the printing apparatus 
1030 for printing the digital zoom information, a single 
apparatus may be provided for realizing the image pick- 
up and the printing. 

[0081] While in this embodiment, the digital zoom in- 
formation is passed via the storage medium 1 020, it may 
be conveyed by providing the communication function 
of the IEEE 1394. 

[0082] While in this embodiment the moving image is 



only treated, the still image may be of course treated. In 
this case, the image quality information corresponding 
to each still image is stored. That is, the process as 
shown in FIGS. 4A and 4B is applied to the still image, 
5 whereby the quality information Q in the zooming oper- 
ation of each still image can be obtained. This invention 
is applicable to the cases where the reproduced image 
quality is different depending on the processing method, 
as a result of processing the data, in which the data may 
10 be sound and so on. 

[0083] As described above, according to this embod- 
iment, for example, even if the same operation is done 
to the image device, the information regarding the dif- 
ference in the image quality as a result of processing 
15 the data, which is caused by the difference in the 
processing method, can be stored and utilized. For ex- 
ample, in the digital zoom, the information regarding the 
digital zoom method and the change in the image quality 
caused through the process is recorded on the record- 
20 ing medium, corresponding to the frame, and utilized by 
the printing apparatus, the reproducing apparatus, the 
editing device, or the retrieval device, thereby inviting 
the user's attention to the frames of bad image quality, 
not displaying the frames of bad image quality, or dis- 
25 playing the frames at several levels depending on the 
image quality. 

[0084] Embodiments of this invention may also com- 
prise a storage medium or signal carrying the program 
code of software for implementing the functions of the 

30 above embodiment. When supplied to the system or ap- 
paratus, the computer (i.e. , CPU or M PU) forthe system 
or apparatus reads and executes the program code 
stored in the storage medium or signal. 
[0085] In this case, the program code itself read from 

35 the storage medium implements the functions of this 
embodiment, and the invention is embodied by the stor- 
age medium or signal carrying the program code. 
[0086] Examples of the storage medium for supplying 
the program code include a floppy disk, a hard disk, an 

40 optical disk, an optical magnetic disk, a CD-ROM, a 
CD-R, amagnetic tape, a non-volatile memory card, and 
ROM. Also, the functions of the above embodiment can 
be implemented not only when the program is read and 
executed by the computer, but also when the OS (Op- 

45 erating System) operating on the computer performs ail 
or part of the actual process, based on the instructions 
of the program code. 

[0087] Moreover, the functions of the embodiment 
may be implemented in such a way that the program 

so code read from the storage medium is written into a 
function extension board inserted into the computer or 
the memory provided in a function extension unit con- 
nected to the computer, and the CPU equipped in the 
function extension board or function extension unit per- 

55 forms all or part of the actual process, based on the in- 
structions of the program code. 

[0088] As described above, with this invention, in the 
case where the data to be reproduced is processed to 
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have some effect on the image quality in reproduction, 
the data is provided with the information regarding the 
image quality in reproduction. 

[0089] Also, with this invention, the apparatus can 
have enhanced operability, employing the information 
regarding the image quality in reproduction. 
[0090] Many apparently widely different embodi- 
ments of the present invention can be made without de- 
parting from the scope thereof, and the invention is not 
limited to the specific embodiments. 



Claims 

1. An information processing apparatus, character- 
ised by comprising: 

processing means (1 6-1 8, 21 , 26) for process- 
ing data intended for reproduction; 
determination means (21 , 21 a) for determining 
a quality of the data for reproduction after 
processing of the data by said processing 
means to generate quality information repre- 
senting its quality; and 

writing means (27) for writing the data proc- 
essed by said processing means and the qual- 
ity, information obtained by said determination 
means in association with each other. 

2. The information processing apparatus according to 
claim 1 , wherein the data is time series data having 
reproduction contents changing dynamically in the 
reproduction with an elapse of time, said determi- 
nation means is operable to determine the quality 
of the data for reproduction for each predetermined 
interval of the data to generate the quality informa- 
tion, and said writing means is operable to write the 
quality information in association with each prede- 
termined interval of the data. 

3. The information processing apparatus according to 
claim 2, wherein the data is moving image data and 
the predetermined interval corresponds to one 
frame. 

4. The information processing apparatus according to 
claim 2, wherein said writing means is operable to 
write period information in association with the qual- 
ity information, the period information indicating a 
period containing a series of intervals having the 
quality information indicating the same quality. 

5. The information processing apparatus according to 
claim 3, wherein said writing means is operable to 
write the quality information by describing the qual- 
ity information in a predetermined area of each 
frame. 



6. The information processing apparatus according to 
claim 3, wherein said writing means is operable to 
write an identifier for identifying each frame in as- 
sociation with the quality information. 

5 

7. The information processing apparatus according to 
claim 1 , wherein the data is still image data, and 
said determination means is operable to determine 
a quality of the data for each still image for repro- 

10 duction to generate the quality information repre- 
senting its quality. 

8. The information processing apparatus according to 
claim 3, wherein said processing means is operable 

15 to perform processing comprising a variable mag- 
nification and reduction power control. 

9. The information processing apparatus according to 
claim 1 , wherein the quality information represents 

20 the quality in reproduction by a quality level in which 
the quality is subdivided into a predetermined 
number of levels. 

10. An information processing apparatus comprising: 

25 

acquisition means (802) for acquiring reproduc- 
tion object data and quality information of each 
portion of the reproduction object data from the 
data including the reproduction object data and 
30 the quality information representing the quality 

of each portion of the reproduction object data 
in reproduction; and 

display control means (803) for controlling a 
display of the information indicating portions of 
35 said reproduction object data based on the 

quality information acquired by said acquisition 
means. 

11. The information processing apparatus according to 
40 claim 1 0, wherein said display control means is op- 
erable to control the display to display information 
indicating each portion of the reproduction object 
data together with information representing the 
quality of each portion of the reproduction object da- 

45 ta. 

12. The information processing apparatus according to 
claim 10, wherein said display control means is op- 
erable to control the display to display the informa- 

50 tion for a portion of the reproduction object data 
whose quality information represents the quality at 
a predetermined level or more. 

13. The information processing apparatus according to 
55 claim 12, further comprising setting means for set- 
ting the predetermined level. 

14. The information processing apparatus according to 



10 



15 



9 



(I 



17 



EP 1 274 241 A2 



18 



claim 10, wherein the reproduction object data is 
moving image data, the quality information repre- 
sents a quality of the moving image data for each 
frame, and said display control means is operable 
to control the display to display a thumb-nail of each 5 
frame of the moving image data together with the 
information indicating the quality information corre- 
sponding to each frame. 

15. The information processing apparatus according to 10 
claim 10, wherein said reproduction object data is 
moving image data, the quality information repre- 
sents the quality of the moving image data for each 
frame, and said display means is operable to control 

the display to display only a thumb-nail of the frame is 
in which the quality information indicates a prede- 
termined level. 

16. An information processing method, characterised 

by comprising: 20 



claim 1 8, wherein the writing step writes the quality 
information by describing the quality information in 
a predetermined area of each frame. 

21. The information processing method according to 
claim 1 8, wherein the writing step comprises writing 
an identifier for identifying each frame in association 
with the quality information. 

22. The information processing method according to 
claim 16, wherein the data is the still image data, 
and the determination step determines the quality 
of the data for each still image for reproduction to 
generate the quality information representing its 
quality. 

23. The information processing method according to 
claim 18, wherein the processing of data involves a 
variable magnification and/or reduction power con- 
trol. 



a processing step (S404-S406, S408, S410, 
S415, S420, S421, and S423) of processing- 
data intended for reproduction; 
a determination step (S404, S405, S407, S409, 
S411 , S415, S420-S422, S424) of determining 
a quality of the data for reproduction after 
processing of the data at the processing step 
to generate quality information representing its 
quality; and 

a writing step (27) of writing the data processed 
at the processing step and the quality informa- 
tion obtained at the determination step in asso- 
ciation with each other. 

17. The information processing method according to 
claim 1 6, wherein the data is time series data having 
reproduction contents changing dynamically in the 
reproduction with the elapse of time, the determina- 
tion step determines the quality of the data for re- 
production for each predetermined interval of the 
data to generate the quality information, and the 
writing step writes the quality information in associ- 
ation with each predetermined interval of the data. 

18. The information processing method according to 
claim 17, wherein the data is moving image data 
and the predetermined interval corresponds to one 
frame. 

19. The information processing method according to 
claim 1 7, wherein the writing step writes period in- 
formation in association with the quality information, 
the period information indicating a series of the pre- 
determined intervals whose quality information in- 
dicates the same quality. 

20. The information processing method according to 



24. The information processing method according to 
claim 1 6, wherein the quality information represents 
the quality in reproduction by a quality level in which 
25 the quality is subdivided into a predetermined 
number of levels. 



30 



35 



40 



25. An information processing method comprising: 

an acquisition step (S1001) of acquiring repro- 
duction object data and quality information of 
each portion of the reproduction object data 
from the data including the reproduction object 
data and quality information representing the 
quality of each portion of the reproduction ob- 
ject data in reproduction; and 
a display control step (S1004-S1006) of con- 
trolling a display of the information indicating 
portions of the reproduction object data based 
on the quality information acquired at the ac- 
quisition step. 



26. The information processing method according to 
claim 25, wherein the display control step controls 
«* a display to display the information indicating each 
portion of the reproduction object data together with 
information representing the quality of each portion 
of the reproduction object data. 

50 27. The information processing method according to 
claim 25, wherein said display control step controls 
the display to display the information for a portion 
of the reproduction object data for which the quality 
information represents the quality at a predeter- 

55 mined level or more. 

28. The information processing method according to 
claim 27, further comprising a setting step of setting 
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the predetermined level. 

29. The information processing method according to 
claim 25, wherein the reproduction object data is the 
moving image data, the quality information repre- 5 
sents the quality of the moving image data of each 
frame, and the display control step controls a dis- 
play to display a thumb-nail of each frame of the 
moving image data together with the information in- 
dicating the quality information corresponding to 10 
each frame. 

30. The information processing method according to 
claim 25, wherein the reproduction object data is the 
moving image data, the quality information repre- 15 
sents the quality of the moving image data of each 
frame, and the display control step controls a dis- 
play to display only a thumb-nail of the frame whose 
quality information indicates a predetermined level. 

20 

31. A control program for a programmable processing 
apparatus comprising instructions for programming 
the apparatus to become operable to implement an 
information processing method according to at least 
one of claims 1 6 to 30. 25 

32. A storage medium storing a control program ac- 
cording to claim 31 . 

33. A signal carrying a control program according to 30 
claim 31. 
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